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Introduction 


The  project  period  encompasses  the  second  phase  of  a  three-phase  initiative  to  install  Picture 
Archiving  and  Communication  Systems  (PACS)  and  Teleradiology  at  North  Mississippi  Medical 
Center’s  (NMMC)  main  hospital  campus  in  Tupelo,  four  community  hospitals  in  northeastern 
Mississippi,  and  at  15  clinics  and  to  test  relevant  research  hypotheses  pertaining  to  quality, 
security,  and  clinical  issues.  Phase  I  involved  1)  installation  of  PACS  equipment  secured  with 
Anny  Security  Architecture  for  Medical  (ARSAM)  Systems  Design  Plan  on  all  freestanding 
buildings  on  the  NMMC  campus,  including  NMMC-Tupelo,  Women’s  Health  Center,  Longtown 
Imaging,  Digestive  Health,  Internal  Medicine  Associates,  and  the  Cancer  Center,  2)  training  of 
NMMC’s  radiologists,  and  3)  design  of  research  database.  Phase  II  deliverables  included  1) 
installation  of  PACS  equipment  secured  with  ARSAM  Systems  Design  Plan  at  four  community 
hospitals  in  northeastern  Mississippi,  2)  training  of  all  radiology  staff  members  at  these  facilities, 

3)  multiple  tests  of  security  architecture,  and  4)  collection  and  analysis  of  efficiency,  satisfaction, 
and  outcome  data. 

Body 

Two  years  ago,  NMMC,  in  conjunction  with  its  health  system,  North  Mississippi  Health  Services 
(NMHS),  proposed  a  three-phase  initiative  to  install  Picture  Archiving  and  Communication 
Systems  (PACS)  and  Teleradiology  at  its  main  hospital  campus,  four  community  hospitals  in 
northeastern  Mississippi,  one  community  hospital  in  northwestern  Alabama,  and  at  15  clinics.  In 
addition  to  improving  the  quality  of  radiology  services  for  people  living  in  this  rural  region, 
including  military  personnel,  military  reserve  personnel,  and  military  dependents,  the  PACS 
systems  would  be  used  to  test  security  architecture  for  networked  medical  devices,  an  initiative 
of  critical  importance  to  the  military,  and  to  research  clinical  outcomes  associated  with  PACS 
technology  that  are  of  significance  to  the  military  and  other  health  care  operations. 

The  hypotheses  to  be  tested  were  that  implementation  of  state-of-the-art  PACS  on  the  NMMC- 
Tupelo  campus,  four  community  hospitals,  and  other  sites  affiliated  with  the  NMHS  system 
will  1)  improve  radiology  efficiency,  2)  improve  clinical  outcomes,  and  3)  demonstrate  that 
networked  PACS  can  be  made  secure  to  intrusion  and  device  compromise. 

Expected  results  of  the  initiative  included: 

1)  Increased  efficiency  of  radiologists  as  measured  by  performance  indicators 

2)  Decreased  number  of  radiology  support  staff  (film  librarians  and  technicians) 

3)  Increased  satisfaction  of  radiologists,  other  medical  providers,  and  patients  as  measured 
by  satisfaction  surveys  (medical  staff,  radiologists,  and  consumers) 

4)  Identification  of  highly  effective  security  architecture  for  networked  PACS  systems  that 
can  be  replicated  for  Army  Medicine’s  networked  medical  devices,  including  PACS 

5)  Improved  clinical  outcomes  as  measured  by  specific  radiology  indicators  on  the  length  of 
stay  (LOS)  and  other  indicators  for  CAP  and  stroke 
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Potential  benefits  of  the  three-phase  project  included: 

•  Demonstration  of  radiology  service  management  at  remote  locations 

•  Information  on  the  impact  of  PACS  technology  on  clinical  outcomes  for  CAP  and  stroke 

•  Demonstration  of  secure  teleradiology  services  at  remote  locations 

The  project’s  relevance  to  the  military  is  related  to  its  capacity  to: 

•  Simulate  the  military’s  management  of  radiology  services  in  remote,  noncombat  settings 

•  Validate  ARSAM  security  architecture  for  PACS 

•  Ensure  availability  of  high  quality  radiological  services  for  military  personnel,  reserve 
personnel,  and  dependents 

Project  Period  Deliverables  and  Results 

All  deliverables  for  this  project  period  were  met.  During  the  period  from  September  29,  2005  to 

April  28,  2006: 

•  PACS  equipment  became  fully  operational  at  all  four  community  hospitals  (Eupora, 
Pontotoc,  West  Point,  and  Iuka  sites) 

•  Radiology  staff  at  all  four  sites  were  thoroughly  training 

•  Data  collection  and  analysis  continued  (see  results  below) 

•  Network  vulnerability  assessment  and  penetration  tests  were  completed  (see  results  below) 

Key  Research  Accomplishments 

>  Research  Area:  Impact  of  PACS  on  efficiency  of  radiologists  as  measured  by 
performance  indicators 

•  Dramatic  reductions  were  noted  in  Report  Turnaround  Time,  the  time  that  elapses 
following  completion  of  the  procedure  and  receipt  of  the  radiologist’s  interpretation  of 
the  image,  pre-  and  post-installation  of  PACS  capabilities.  At  the  hospitals  in  Eupora  and 
Pontotoc,  the  wait  for  interpretations  decreased  from  averages  of  17  hours  to  less  than 
two  hours.  Refer  to  Figure  1  on  following  page. 
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Figure  1 

Report  Turnaround  Time 

(From  completion  of  procedures  to  radiologists’  interpretations) 

(March  2004  thru  August  2004  versus  March  2005  thru  August  2005) 


Turnaround  Time 
(March  2004  thru 
August  2004) 

Turnaround  Time 
(March  2005  thru 
August  2005) 

Pontotoc 

13  hours 

.5  hours 

West  Point 

9  hours 

1.8  hours 

Eupora 

17  hours 

1 .3  hours 

Iuka 

17  hours 

1.8  hours 

>  Research  Area:  Impact  of  PACS  on  costs  for  radiology  support  staff  (film 
librarians  and  technicians)  and  film  (Figure  2) 

•  Two  of  the  four  hospitals  experienced  cost  savings  as  a  result  of  PACS  implementation. 
Radiology  technicians  at  Pontotoc  and  Eupora  also  doubled  as  film  librarians,  so  their 
positions  were  maintained  when  PACS  was  introduced. 

Figure  2 
Cost  Reduction 

(February  2005  through  January  2006) 


Film  Cost  Reduction 

Personnel  Reduction/ 
Salary  Savings 

Pontotoc 

West  Point 

Eupora 

Iuka 

>  Research  Area:  Impact  of  PACS  on  satisfaction  of  radiologists,  other  medical 
providers,  and  patients  as  measured  by  satisfaction  surveys  (medical  staff, 
radiologists,  and  consumers) 

a.  Referring  Physician  Survey 

•  In  each  of  the  four  categories  of  satisfaction  questions,  the  cumulative  score  increased 
from  the  baseline  survey  in  2003  to  post-PACS  in  2006.  Refer  to  Figure  3. 

•  Physician  response  rate  increased  from  1 1.8  percent  in  2003  to  33.9  percent  in  2006 
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Figure  3  -  Referring  Physician  Survey  Results 


Referring  Physicians 

2003 

2006 

Scheduling  &  Registration 

3.65 

4.33 

Time  it  takes  to  schedule  outpatients 

-  4.25 

Responsiveness  for  urgent  patients 

-  4.42 

Physician  Needs 

3.74 

4.15 

Availability  of  radiologists  to  review  films 

-  4.36 

Timeliness  of  receiving  preliminary  reports 

-  3.97 

Timeliness  of  receiving  final  reports 

-  4.01 

Timeliness  in  completion  of  urgent  exams 

-  4.29 

Report  turn-around  time  for  urgent  patients 

-  4.14 

Patient  Needs 

3.72 

3.97 

Waiting  time  in  the  department 

-  3.57 

Customer  service  attitude  &  tech  perfonnance 

-  4.37 

Radiologists 

3.93 

4.16 

Quality  &  accuracy  of  interpretation 

-  4.20 

Availability  for  consultation 

-  4.15 

Calling  of  positive  reports  to  physicians 

-  4.15 

•  To  supplement  the  empirical  data,  which  trends  strongly  to  greater  satisfaction  with 

radiology  services,  physician  respondents  write-in  comments  are  presented  in  Figure  4. 

Figure  4 

Referring  Physician  Survey  Write-in  Comments  (2006) 


Do  you  feel  that  the  implementation  of  PACS  has  changed  the  way  you  treat 
patients?  For  example,  if  the  turn  around  time  for  reports  is  faster,  has  this  helped 
with  your  ability  to  diagnose  and  treat  patients? _ 

•  Greatly  improved. 

•  No  change. 

•  Yes,  I  can  pull  of  the  films  and  look  myself. 

•  Much  easier. 

•  We  do  not  have  PACS.  (NMMCI  physician) 

•  Very  good  service.  Able  to  review  exams  very  quickly  without  having  to  wait  for 
film  processing.  Do  not  have  to  deal  with  films  in  office.  (NMMCI  physician) 

•  Turnaround  is  faster. 

•  I  personally  do  not  use  PACS.  (NMMCI  staff  physician) 

•  Patients  return  to  the  office  before  reports  are  available,  even  in  PACS.  (Private 
physician) 

•  Faster,  convenient  to  review  films  and  reports  from  many  locations. _ 
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Do  you  feel  that  the  implementation  of  PACS  has  changed  the  way  you  treat 

patients?  For  example,  if  the  turn  around  time  for  reports  is  faster,  has  this  helped 

with  your  ability  to  diagnose  and  treat  patients? _ 

•  The  faster,  the  better. 

•  We  are  not  on  PACS  yet.  (NMMCI  physician) 

•  Much  faster,  especially  in  emergency  department. 

•  Quicker  and  more  efficient  to  view  films. 

•  Easier  to  read  images  off  of  computer. 

•  I  have  not  had  to  fight  with  the  file  room  personnel. 

•  Not  really  a  change.  (NMMCI  staff  physician) 

•  Great  system. 

•  Able  to  review  tests  more  quickly  and  make  treatment  decisions  a  little  more  quickly. 

•  Has  greatly  helped  with  timeliness  of  patient  care. 

•  This  would  be  a  great  way  to  improve  turnaround  time.  We  could  actually  have  the 
report  the  same  day  the  patient  is  here.  (NMMCI  physician) 

•  Yes,  am  now  able  to  pull  up  old  x-rays  to  compare  with  new  x-rays. 

•  PACS  in  the  NICU  has  been  great  for  expediting  care  of  patients  and  accomplishing 

routine  rounds,  including  consultation  and  teaching  with  the  entire  staff  -  when  it 
works,  which  is  almost  all  of  the  time  -  but  when  there  are  problems  viewing  films 
lost  in  the  system,  it  is  a  disaster,  greatly  confounding  care.  (Private  practice  - 
Women’s  Hospital  NMMC  -  NICU) _ 


Comments  on  Referring  Physician  Survey: 

Note  that  several  NMMCI  physicians  stated  that  they  do  not  use  PACS.  The  survey  was 
designed  for  responding  to  PACS  services  at  NMMC-Tupelo  and  the  four  community  hospitals; 
these  NMMCI  physicians  work  in  the  clinic  system,  which  will  receive  PACS  during  the  current 
project  period.  They  will  be  resurveyed  specifically  about  the  impact  of  PACS  on  their  practice. 

b.  Patient  Satisfaction  Survey 

•  NMHS  switched  its  patient  satisfaction  survey  company  from  PRC  to  Press-Ganey  in 

2005.  Unfortunately,  only  one  month  (March  2005)  of  baseline  patient  satisfaction  data 
with  outpatient  radiation  services  is  available  for  just  three  (Pontotoc,  West  Point  and 
Eupora)  of  the  four  community  hospitals.  The  number  of  satisfaction  surveys  is  low  in 
March  2005  because  a  new  survey  system  was  being  initiated.  Based  on  these  few 
surveys,  it  appears  that  patient  satisfaction  with  the  pre-PACS  service  was  high  and 
remains  high  (Figure  5). 


Figure  5  -  Patient  Satisfaction  Survey  Results 


MONTH 

N:  2005 

N: 2006 

%  Patients  Very 
Satisfied  2005 

%  Patients  Very 
Satisfied  2006 

March 

7 

55 

89.5 

88.7 

April 

18 

40 

91.1 

89.7 

May 

28 

53 

87.4 

93.1 

June 

23 

82.9 

July 

24 

92.9 

August 

26 

92.5 

September 

23 

91.6 

>  Research  Area:  Improved  clinical  outcomes  as  measured  by  specific  radiology 
indicators  on  the  length  of  stay  (LOS)  and  other  indicators  for  CAP  and  stroke. 

•  At  its  community  hospitals  in  Eupora,  Pontotoc,  and  West  Point,  NMMC  is  monitoring 
clinical  outcomes  as  they  relate  to  the  implementation  of  PACS  for  two  conditions: 
community-acquired  pneumonia  (CAP)  and  ischemic  stroke,  which  were  selected  due  to 
their  high  base  rates  of  incidence  among  admissions  to  community  hospitals  and  because 
diagnosis  and  initial  treatments  of  CAP  and  ischemic  stroke  are  dependent  upon 
radiological  studies. 

•  Reviewers  of  the  Phase  1  portion  of  this  project  (implementing  PACS  on  the  NMMC- 
Tupelo  campus)  recommended  incorporation  of  process  indicators  that  were  relatively 
close  to  and  dependent  on  PACS  to  more  reliably  assess  its  impact.  The  timing  of  the 
first  dose  of  antibiotic  was  selected  as  the  process  indicator  for  CAP  and  the 
administration  of  tissue  plasminogen  activator  (tPA)  as  the  process  indicator  for  ischemic 
stroke.  As  in  Phase  1,  overall  outcome  indicators  (length  of  stay,  mortality  and  cost) 
were  also  measured  pre-  and  post-PACS  implementation 

•  Refer  to  Figure  6  on  the  following  page.  NMMC-Eupora  had  the  most  patients 
diagnosed  with  CAP  pre-  and  post-PACS  (even  with  a  29%  decline  in  the  post¬ 
intervention  period).  This  patient  population  also  was  the  most  similar  (age  and  severity 
ranking)  between  the  two  assessment  periods.  During  this  timeframe,  the  radiologists’ 
turnaround  times  dropped  from  17  hours  to  1.3  hours  (88  minutes),  and  the  median  time 
from  emergency  department  admission  to  administering  antibiotics  dropped  from  176  to 
98  minutes.  This  is  well  below  the  four  hour  (240  minute)  goal  for  antibiotic 
administration  and  may  have  contributed  to  the  slight  decrease  in  LOS,  5.81  to  5.47  days 
(5.8%)  and  the  significant  decrease  in  mortality  (3.96%  to  1.39%  (65%  decrease)).  The 
median  cost  of  the  admission  also  dropped  slightly,  from  —  to  — . 

•  The  other  two  hospitals,  West  Point  and  Pontotoc,  also  had  positive  LOS,  mortality  and 
cost  outcomes,  but  their  patient  populations  were  different  (age  and  severity)  between  the 
two  time  periods  and  their  change  in  median  antibiotic  administration  time  was  not 
significantly  different.  West  Point  did  not  demonstrate  any  change  in  its  median  time  to 
antibiotic  administration.  This  facility  has  regular  daytime  on-site  radiologist  coverage 
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and  experienced  the  least  improvement  in  radiologist  study  turn-around  time  (9.0  to  1.8 
hours:  Figure  1).  The  regular  presence  of  a  radiologist  may  account  for  the  lack  of 
improvement  on  antibiotic  administration,  i.e.,  the  availability  of  PACS  did  not  change 
their  practice  sufficiently.  In  short,  it  is  difficult  to  project  if  the  implementation  of 
PACS  may  have  been  associated  with  these  positive  outcomes. 

Figure  6 

CAP  Outcome  Analysis  Table 


Median  Ave. 

time  to  length 


Number 

Ave. 

Ave.  pt 

antibiotic 

of  stay 

Patient 

Median 

EUPORA 

of  pts. 

pt  age 

severity 

(mins) 

(days) 

mortality 

Cost 

Pre-PACS  (April-Sept  2004) 

101 

67 

2.09 

176 

5.81 

3.96% 

Post-PACS  (April-Sept  2005) 

72 

61 

2.04 

98 

5.47 

1.39% 

PONTOTOC 


Pre-PACS  (April-Sept  2004) 

17 

72 

1.81 

120 

4.29 

0.00% 

Post-PACS  (April-Sept  2005) 

37 

65 

2.19 

115 

3.84 

0.00% 

WEST  POINT 


Pre-PACS  (April-Sept  2004) 

51 

66 

2.08 

120 

6.67 

7.84% 

Post-PACS  (April-Sept  2005) 

45 

49 

1.91 

120 

4.13 

0.00% 

•  Refer  to  Figure  7  on  the  following  page.  Only  one  hospital,  West  Point,  administered 
tPA  to  one  patient.  It  was  during  the  post-PACS  timeframe,  but  as  noted  earlier,  West 
Point  already  had  about  40  hours  per  week  of  on-site  radiologist  time,  so  it  is  impossible 
to  attribute  this  tPA  usage  to  the  implementation  of  PACS.  Although  the  numbers  of 
stroke  patients  are  low,  based  on  this  6-month  pre-  and  post-  PACS  implementation 
analysis,  it  does  not  appear  that  PACS  has  made  an  impact  on  the  care  of  stroke  patients 
with  regard  to  tPA  administration,  the  chosen  process  marker.  Whereas  mortality 
decreased  in  all  three  hospitals,  the  LOS  and  median  costs  of  care  decreased  in  the  two 
hospitals  that  did  not  administer  the  tPA. 
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Figure  7 


Stroke  Outcome  Analysis  Table 


Ave. 

Ave. 

Ave. 

tPA 

length 

Patient 

Number 

pt 

sev 

adminis¬ 

of  stay 

mortalit 

Median 

of  pts. 

age 

ind 

tration 

(days) 

y 

Cost 

EUPORA 


Pre-PACS  (April-Sept  2004) 

16 

79 

2.25 

0 

5.69 

6.30% 

Post-PACS  (April-Sept  2005) 

15 

82 

2 

0 

3.56 

0.00% 

II  KA 


Pre-PACS  (July-Dee  2004) 

4 

71 

2.5 

0 

3.75 

25.00% 

Post-PACS  (July-Dee  2005) 

13 

69 

2.5 

0 

3.93 

7.10% 

WEST  POINT 


Pre-PACS  (April-Sept  2004) 

16 

74 

2.38 

0 

5.00 

6.30% 

Post-PACS  (April-Sept  2005) 

19 

68 

2.11 

1 

6.47 

0.00% 

Project  Period  Security  Architecture  Testing 

>  Research  Area:  Identification  of  highly  effective  security  architecture  for 

networked  PACS  systems  that  can  be  replicated  for  Army  Medicine’s  networked 
medical  devices,  including  PACS 

•  NMMC  contracted  Sword  &  Shield  Enterprise  Security,  Inc.  (Sword  &  Shield)  to 
conduct  a  network  vulnerability  assessment  and  penetration  test  of  NMMC’s  external  and 
internal  networks.  The  assessment  examined  NMMC's  implementation  of  the  Anny 
Security  Architecture  for  Medical  (ARSAM),  a  defense-in-depth  network  security 
architecture  for  FDA-approved  medical  devices.  NMMC  is  using  ARSAM  in  its 
deployment  of  the  PACS  within  its  hospital  information  system  network. 

•  From  an  external  perspective,  the  overall  security  posture  of  the  NMMC  network  was 
deemed  to  be  satisfactory.  The  firewall  and  network  devices  provided  an  adequate  level 
of  protection  to  the  NMMC  internal  network,  and  PACS  network,  from  Internet-based 
threats. 

•  However,  Sword  and  Shield  was  able  to  circumvent  the  ARSAM  protections 
implemented  at  NMMC  during  the  internal  phase  of  testing  by  taking  advantage  of  some 
PACS  devices  that  were  located  outside  of  the  protected  enclave  and  also  by  taking 
advantage  of  other  devices  on  the  network  which  were  not  patched  with  all  of  the  latest 
security  patches  or  system/applications  updates. 
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Reportable  Outcomes 


To  date,  no  manuscripts,  abstracts,  grant  applications,  patents,  etc.  have  been  submitted  that  are 
direct  results  of  this  project. 

Conclusions 

Implementation  of  PACS  technology  on  NMMC’s  campus  in  Tupelo  and  four  community 
hospitals  has  yielded  information  of  potential  value  to  other  health  care  agencies  serving  rural 
regions: 

1 .  PACS  technology  is  associated  with  dramatic  decreases  in  radiological  report  turn¬ 
around  in  rural  settings 

2.  At  two  of  the  four  sites  (50  percent),  the  introduction  of  PACS  technology  resulted  in 
cost  savings  (personnel  and  film) 

3.  Physicians  reported  greater  satisfaction  with  radiological  services  following  introduction 
of  PACS  capabilities 

4.  Due  to  a  change  in  the  patient  satisfaction  survey  tool,  there  is  not  enough  reliable  pre- 
PACS  patient  satisfaction  data  to  determine  if  PACS  has  an  impact  on  patient 
satisfaction. 

5.  Data  analyzed  to  date  does  not  support  the  hypothesis  that  the  use  of  PACS  technology 
increases  the  speed  with  which  antibiotics  are  administered  as  indicated  for  CAP. 

6.  The  low  usage  of  tPA  in  the  management  of  ischemic  stroke  makes  it  an  unreliable 
process  indicator  for  efficacy  of  PACS  in  the  management  of  stroke. 

7.  Overall  outcomes  (length  of  stay,  patient  mortality  and  median  cost)  trended  towards 
improvement  in  the  post-PACS  period  for  both  CAP  and  stroke. 

8.  Network  sensitivity  and  penetration  testing  revealed  that  ARSAM  protections 
implemented  at  NMMC  were  inadequate  and  that  additional  precautions  needed  to  be 
taken  in  the  security  system  architecture 

As  required  by  the  Research  Technical  Reporting  Requirements,  “So  what?”  Work  completed 
during  the  first  two  phases  of  PACS  implementation  and  research  at  NMMC  does  indeed  have 
implications  for  rural  hospitals/health  systems  and  for  the  military.  PACS  technology  has 
dramatically  increased  the  tum-around  time  for  radiological  images  and  increased  significantly  the 
number  of  “first  reads”  made  by  radiologists  rather  than  other  physician  specialists.  While  this 
speed  may  not  impact  indicated  treatments  (process  indicators)  for  CAP  or  stroke,  it  may  be 
associated  with  the  trend  towards  improved  outcomes  in  this  study  and/or  may  result  in  more 
positive  process  indicators  and  outcomes  for  other  clinical  diagnoses  not  currently  under  study. 
PACS  technology  will  likely  be  instrumental  in  improving  physician  satisfaction  with  radiological 
services  at  other  rural  hospitals,  which  struggle  to  recruit  and  retain  qualified  health  care  providers. 
When  contemplating  implementation  of  PACS,  some  hospitals  may  be  able  to  project  cost  savings 
from  personnel  or  supply  (film)  budgets.  The  results  of  network  sensitivity  and  penetration  testing 
completed  to  date  at  NMMC  suggest  that  ARSAM  protections  must  be  enhanced  to  guard  patient 
data,  which  will  guide  other  rural  hospitals  in  the  implementation  and  design  of  their  PACS 
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systems.  Test  results  are  also  of  significance  to  the  military’s  efforts  to  deliver  radiological  services 
from  remote,  noncombat  sites  to  battlespaces. 

The  health  system’s  experiences  will  be  used  to  formulate  recommendations  for  installation, 
implementation,  and  application  of  PACS  systems,  including  highly  effective  security 
architecture.  Data  collected  on  indicators  pertinent  to  clinical  outcomes  will  be  used  to 
document  the  impact  of  PACS  technology  in  civilian  applications. 
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